Evolution of Spiking Neural Controllers for Autonomous Vision-Based Robots

We describe a set of preliminary experiments to evolve spiking neural controllers for a vision-based mobile robot. All the evolutionary experiments are carried out on physical robots without human intervention. After discussing how to implement and interface these neurons with a physical robot, we show that evolution finds relatively quickly functional spiking controllers capable of navigating in irregularly textured environments without hitting obstacles using a very simple genetic encoding and fitness function. Neuroethological analysis of the network activity let us understand the functioning of evolved controllers and tell the relative importance of single neurons independently of their observed firing rate. Finally, a number of systematic lesion experiments indicate that evolved spiking controllers are very robust to synaptic strength decay that typically occurs in hardware implementations of spiking circuits. 1 Spiking Neural Circuits The great majority of biological neurons communicate by sending pulses along the axons to other neurons. A pulse is a small current charge that occurs when the voltage potential across the membrane of a neuron exceeds its threshold. The pulse is also known as “spike” to indicate its short and transient nature. After emitting a spike, a neuron needs some time to reestablish its electrochemical equilibrium and therefore cannot immediately emit a new spike, no matter how strong its excitatory input is. A typical neuron in the cortex “fires” approximately 10 spikes per second during resting conditions and can emit up to 300 spikes per second in operating conditions. Other neurons can fire more frequently (for example 500 spikes per seconds) clustered in short periods of time (“bursting neurons”). In the field of artificial neural networks we find two different classes of models that differ with respect to the interpretation of the role of spikes. Connectionist models of neural networks [19], by far the most widespread models, assume that what matters in the communication among neurons is the firing rate of a neuron. The firing rate is the average quantity of spikes emitted by the neuron over a relatively long time window (for example, over 100 ms). This quantity is represented by the activation level of the neuron. For example, a neuron characterized by a sigmoid activation function, such as the logistic function f(x) = 1/1+ exp(−x), T. Gomi (Ed.): ER 2001, LNCS 2217, pp. 38–61, 2001. Evolution of Spiking Neural Controllers 39 that gives an output of 0.5 would be equivalent to a spiking neuron that emits approximately half its maximum rate of spikes (150 out of 300 spikes per second, e.g.). Models of pulsed neural networks [14] instead assume that the firing time, that is the precise time of emission of a single spike, can transmit important information for the post-synaptic neuron [23]. Therefore, these models use more complex activation functions that simulate the emission and reception of spikes on a very fine timescale. Spiking circuits have at least two properties that make them interesting candidates for adaptive control of autonomous behavioral robots: – The intrinsic time-dependent dynamics of neuron activation could detect and exploit more easily (e.g., with simpler circuits or with higher reliability) temporal patterns of sensory-motor events than connectionist neural networks. – Since the physics of circuits of sub-threshold transistors (i.e., characterized by gate-to-source voltage differences below their threshold voltage) implemented with analog Very Large Scale Integration technology [15] match the properties of spiking neurons, it possible to implement large networks of spiking neurons in tiny and low-power chips [9]. Designing circuits of spiking neurons with a given functionality is still a challenging task and the most successful results obtained so far have focused on the first stages of sensory processing and on simple motor control. For example, Indiveri et al. [8] have developed neuromorphic vision circuits that emulate the interconnections among the neurons in the early layers of an artificial retina in order to extract motion information and a simple form of attentive selection of visual stimuli. These vision circuits have been interfaced with a Koala robot and their output has been used to drive the wheels of the robot in order to follow lines [10]. In another line of work, Lewis et al. have developed an analog VLSI circuit with four spiking neurons capable of controlling a robotic leg and adapting the motor commands using sensory feedback [13]. This neuromorphic circuit consumes less than 1 microwatt of power and takes less than 0.4 square millimeters of chip area. Despite these interesting implementations, there are not yet methods for developing complex spiking circuits that could display minimally-cognitive functions or learn behavioral abilities through autonomous interaction with the environment. Furthermore, the potentially complex dynamics that a spiking circuit with feedback loops can display allows several alternative functioning modalities. For example, a spiking circuit could use inhibitory neurons to modulate the overall excitation of the network and/or selectively interact with other neurons to inhibit specific actions. Also, the same circuit could use firing rate instead of firing time as the preferred functioning modality (this concept will be discussed in more detail later on in this paper). Artificial evolution is therefore an interesting method to discover spiking circuits that autonomously develop desired behavioral abilities for robots without imposing constraints on their architecture and functioning modality. In this paper, we describe some initial explorations in the evolution of spiking circuits for 40 D. Floreano and C. Mattiussi Fig. 1. The function describing the contribution of a spike from a presynaptic neuron emitted at time t . The contribution of the spike begins after some delay ∆ (2 ms) due to the traveling time of the spike and eventually decreases its effect as time t flows from the firing time t . The synapse time constant τs is set to 10 ms and the membrane time constant τm is set to 4 ms. a task of vision-based navigation using a Khepera robot with a linear CMOS vision system. We will then analyze the evolved spiking circuit and discuss the most important lessons learned from these experiments. Finally, we will describe some ideas for future developments in this emerging area of research. 2 The Spike Response Model The state of a spiking neuron is described by the voltage difference across its membrane, also known as membrane potential υ. Incoming spikes can increase or decrease the membrane potential. The neuron emits a spike when the total amount of excitation induced by incoming excitatory and inhibitory spikes exceeds its firing threshold θ. After firing, the membrane potential of the neuron resets its state to a low negative voltage during which it cannot emit a new spike, and gradually returns to its resting potential. This recharging period is called the refractory period. There are several models of spiking neurons that account for these properties with various degrees of detail. In the experiments described in this paper, we have chosen the Spike Response Model developed by Gerstner [5]. It has been shown that several other models of spiking neurons, such as the class of Integrate-andFire neurons (where the membrane potential of the neuron is immediately reset to its resting value after a spike), represent special cases of the Spike Response Model [6]. In this model, the effect of an incoming spike on the neuron membrane is a function of the difference s = t − t between current time t and the time when the spike was emitted t . The properties of the function are determined by a) the delay ∆ between the generation of a spike at the pre-synaptic neuron Evolution of Spiking Neural Controllers 41 Fig. 2. The function describing the refractory period of the neuron after emission of a spike. Notice the at time 0, the membrane potential is set to a large negative value (larger than 1.0 shown on the y-axis) to prevent another spike. The refractory period is a function of time from last spike and its shape is given by the membrane time constant